The Use of Machine Learning in Eye Tracking Studies in Medical Imaging: A Review.

Machine learning (ML) has revolutionized medical image-based diagnostics. In this review, we cover a rapidly emerging field that can be potentially significantly impacted by ML - eye tracking in medical imaging. The review investigates the clinical, algorithmic, and hardware properties of the existing studies. In particular, it evaluates 1) the type of eye-tracking equipment used and how the equipment aligns with study aims; 2) the software required to record and process eye-tracking data, which often requires user interface development, and controller command and voice recording; 3) the ML methodology utilized depending on the anatomy of interest, gaze data representation, and target clinical application. The review concludes with a summary of recommendations for future studies, and confirms that the inclusion of gaze data broadens the ML applicability in Radiology from computer-aided diagnosis (CAD) to gaze-based image annotation, physicians' error detection, fatigue recognition, and other areas of potentially high research and clinical impact.

Special Section Editorial: Artificial Intelligence for Medical Imaging in Clinical Practice.

The editorial introduces the JMI Special Section on Artificial Intelligence for Medical Imaging in Clinical Practice.

Assessing satisfaction of search in virtual mammograms for experienced and novice searchers.

Purpose: Satisfaction of search (SOS) is a phenomenon where searchers are more likely to miss a lesion/target after detecting a first lesion/target. Here, we investigated SOS for masses and calcifications in virtual mammograms with experienced and novice searchers to determine the extent to which: (1) SOS affects breast lesion detection, (2) similarity between lesions impacts detection, and (3) experience impacts SOS rates. Approach: The open virtual clinical trials framework was used to simulate the breast anatomy of patients, and up to two simulated masses and/or single-calcifications were inserted into the breast models. Experienced searchers (residents, fellows, and radiologists with breast imaging experience) and novice searchers (undergraduates who had no breast imaging experience) were instructed to search for up to two lesions (masses and calcifications) per image. Results: 2×2 mixed factors analysis of variances (ANOVAs) were run with: (1) single versus second lesion hit rates, (2) similar versus dissimilar second-lesion hit rates, and (3) similar versus dissimilar second-lesion response times as within-subject factors and experience as the between subject's factor. The ANOVAs demonstrated that: (1) experienced and novice searchers made a significant amount of SOS errors, (2) similarity had little impact on experienced searchers, but novice searchers were more likely to miss a dissimilar second lesion compared to when it was similar to a detected first lesion, (3) experienced and novice searchers were faster at finding similar compared to dissimilar second lesions. Conclusions: We demonstrated that SOS is a significant cause of lesion misses in virtual mammograms and that reader experience impacts detection rates for similar compared to dissimilar abnormalities. These results suggest that experience may impact strategy and/or recognition with theoretical implications for determining why SOS occurs.

Clinical applications of artificial intelligence in radiology.

The rapid growth of medical imaging has placed increasing demands on radiologists. In this scenario, artificial intelligence (AI) has become an attractive partner, one that may complement case interpretation and may aid in various non-interpretive aspects of the work in the radiological clinic. In this review, we discuss interpretative and non-interpretative uses of AI in the clinical practice, as well as report on the barriers to AI's adoption in the clinic. We show that AI currently has a modest to moderate penetration in the clinical practice, with many radiologists still being unconvinced of its value and the return on its investment. Moreover, we discuss the radiologists' liabilities regarding the AI decisions, and explain how we currently do not have regulation to guide the implementation of explainable AI or of self-learning algorithms.

Fifty years of SPIE Medical Imaging proceedings papers.

Purpose: To commemorate the 50th anniversary of the first SPIE Medical Imaging meeting, we highlight some of the important publications published in the conference proceedings. Approach: We determined the top cited and downloaded papers. We also asked members of the editorial board of the Journal of Medical Imaging to select their favorite papers. Results: There was very little overlap between the three methods of highlighting papers. The downloads were mostly recent papers, whereas the favorite papers were mostly older papers. Conclusions: The three different methods combined provide an overview of the highlights of the papers published in the SPIE Medical Imaging conference proceedings over the last 50 years.

Look how far we have come: BREAST cancer detection education on the international stage.

The development of screening mammography over 30 years has remarkedly reduced breast cancer-associated mortality by 20%-30% through detection of small cancer lesions at early stages. Yet breast screening programmes may function differently in each nation depending on the incidence rate, national legislation, local health infrastructure and training opportunities including feedback on performance. Mammography has been the frontline breast cancer screening tool for several decades; however, it is estimated that there are 15% to 35% of cancers missed on screening which are owing to perceptual and decision-making errors by radiologists and other readers. Furthermore, mammography screening is not available in all countries and the increased speed in the number of new breast cancer cases among less developed countries exceeds that of the developed world in recent decades. Studies conducted through the BreastScreen Reader Assessment Strategy (BREAST) training tools for breast screening readers have documented benchmarking and significant variation in diagnostic performances in screening mammogram test sets in different countries. The performance of the radiologists from less well-established breast screening countries such as China, Mongolia and Vietnam were significant lower in detecting early-stage cancers than radiologists from developed countries such as Australia, USA, Singapore, Italy. Differences in breast features and cancer presentations, discrepancies in the level of experiences in reading screening mammograms, the availability of high-quality national breast screening program and breast image interpretation training courses between developed and less developed countries are likely to have impact on the variation of readers' performances. Hence dedicated education training programs with the ability to tailor to different reader cohorts and different population presentations are suggested to ameliorate challenges in exposure to a range of cancer cases and improve the interpretation skills of local radiologists. Findings from this review provide a good understanding of the radiologist' performances and their improvement using the education interventions, primarily the BREAST program, which has been deployed in a large range of developing and developed countries in the last decade. Self-testing and immediate feedback loops have been shown to have important implications for benchmarking and improving the diagnostic accuracy in radiology worldwide for better breast cancer control.

Special Section Guest Editorial: Conclusion to the Special Series on 2D and 3D Imaging: Perspectives in Human and Model Observer Performance.

Guest editors Claudia Mello-Thoms, Craig K. Abbey, and Elizabeth A. Krupinski conclude the JMI Special Series on 2D and 3D Imaging, with commentary on the contributions.

Introducing the Special Series on 2D and 3D Imaging: Perspectives in Human and Model Observer Performance.

Guest editors Claudia Mello-Thoms, Craig Abbey, and Elizabeth A. Krupinski introduce the Special Series on 2D and 3D Imaging: Perspectives in Human and Model Observer Performance.

Can a Machine Learn from Radiologists' Visual Search Behaviour and Their Interpretation of Mammograms-a Deep-Learning Study.

Visual search behaviour and the interpretation of mammograms have been studied for errors in breast cancer detection. We aim to ascertain whether machine-learning models can learn about radiologists' attentional level and the interpretation of mammograms. We seek to determine whether these models are practical and feasible for use in training and teaching programmes. Eight radiologists of varying experience levels in reading mammograms reviewed 120 two-view digital mammography cases (59 cancers). Their search behaviour and decisions were captured using a head-mounted eye-tracking device and software allowing them to record their decisions. This information from radiologists was used to build an ensembled machine-learning model using top-down hierarchical deep convolution neural network. Separately, a model to determine type of missed cancer (search, perception or decision-making) was also built. Analysis and comparison of variants of these models using different convolution networks with and without transfer learning were also performed. Our ensembled deep-learning network architecture can be trained to learn about radiologists' attentional level and decisions. High accuracy (95%, p value ≅ 0 [better than dumb/random model]) and high agreement between true and predicted values (kappa = 0.83) in such modelling can be achieved. Transfer learning techniques improve by < 10% with the performance of this model. We also show that spatial convolution neural networks are insufficient in determining the type of missed cancers. Ensembled hierarchical deep convolution machine-learning models are plausible in modelling radiologists' attentional level and their interpretation of mammograms. However, deep convolution networks fail to characterise the type of false-negative decisions.

Visual search in breast imaging.

Breast cancer is the most common cancer among females worldwide and large volumes of breast images are produced and interpreted annually. As long as radiologists interpret these images, the diagnostic accuracy will be limited by human factors and both false-positive and false-negative errors might occur. By understanding visual search in breast images, we may be able to identify causes of diagnostic errors, find ways to reduce them, and also provide a better education to radiology residents. Many visual search studies in breast radiology have been devoted to mammography. These studies showed that 70% of missed lesions on mammograms attract radiologists' visual attention and that a plethora of different reasons, such as satisfaction of search, incorrect background sampling, and incorrect first impression can cause diagnostic errors in the interpretation of mammograms. Recently, highly accurate tools, which rely on both eye-tracking data and the content of the mammogram, have been proposed to provide feedback to the radiologists. Improving these tools and determining the optimal pathway to integrate them in the radiology workflow could be a possible line of future research. Moreover, in the past few years deep learning has led to improving diagnostic accuracy of computerized diagnostic tools and visual search studies will be required to understand how radiologists interact with the prompts from these tools, and to identify the best way to utilize them. Visual search in other breast imaging modalities, such as breast ultrasound and digital breast tomosynthesis, have so far received less attention, probably due to associated complexities of eye-tracking monitoring and analysing the data. For example, in digital breast tomosynthesis, scrolling through the image results in longer trials, adds a new factor to the study's complexity and makes calculation of gaze parameters more difficult. However, considering the wide utilization of three-dimensional imaging modalities, more visual search studies involving reading stack-view examinations are required in the future. To conclude, in the past few decades visual search studies provided extensive understanding about underlying reasons for diagnostic errors in breast radiology and characterized differences between experts' and novices' visual search patterns. Further visual search studies are required to investigate radiologists' interaction with relatively newer imaging modalities and artificial intelligence tools.

Characteristics of Mammographic Breast Density and Associated Factors for Chinese Women: Results from an Automated Measurement.

BACKGROUND: Characteristics of mammographic density for Chinese women are understudied. This study aims to identify factors associated with mammographic density in China using a quantitative method. METHODS: Mammographic density was measured for a total of 1071 (84 with and 987 without breast cancer) women using an automatic algorithm AutoDensity. Pearson tests examined relationships between density and continuous variables and t-tests compared differences of mean density values between groupings of categorical variables. Linear models were built using multiple regression. RESULTS: Percentage density and dense area were positively associated with each other for cancer-free (r=0.487, p<0.001) and cancer groups (r=0.446, p<0.001), respectively. For women without breast cancer, weight and BMI (p<0.001) were found to be negatively associated (r=-0.237, r=-0.272) with percentage density whereas they were found to be positively associated (r=0.110, r=0.099) with dense area; age at mammography was found to be associated with percentage density (r=-0.202, p<0.001) and dense area (r=-0.086, p<0.001) but did not add any prediction within multivariate models; lower percentage density was found within women with secondary education background or below compared to women with tertiary education. For women with breast cancer, percentage density demonstrated similar relationships with that of cancer-free women whilst breast area was the only factor associated with dense area (r=0.739, p<0.001). CONCLUSION: This is the first time that mammographic density was measured by a quantitative method for women in China and identified associations should be useful to health policy makers who are responsible for introducing effective models of breast cancer prevention and diagnosis.

Computer-Assisted Nuclear Atypia Scoring of Breast Cancer: a Preliminary Study.

Inter-pathologist agreement for nuclear atypia scoring of breast cancer is poor. To address this problem, previous studies suggested some criteria for describing the variations appearance of tumor cells relative to normal cells. However, these criteria were still assessed subjectively by pathologists. Previous studies used quantitative computer-extracted features for scoring. However, application of these tools is limited as further improvement in their accuracy is required. This study proposes COMPASS (COMputer-assisted analysis combined with Pathologist's ASSessment) for reproducible nuclear atypia scoring. COMPASS relies on both cytological criteria assessed subjectively by pathologists as well as computer-extracted textural features. Using machine learning, COMPASS combines these two sets of features and output nuclear atypia score. COMPASS's performance was evaluated using 300 images for which expert-consensus derived reference nuclear pleomorphism scores were available, and they were scanned by two scanners from different vendors. A personalized model was built for three pathologists who gave scores to six atypia-related criteria for each image. Leave-one-out cross validation (LOOCV) was used. COMPASS was trained and tested for each pathologist separately. Percentage agreement between COMPASS and the reference nuclear scores was 93.8%, 92.9%, and 93.1% for three pathologists. COMPASS's performance in nuclear grading was almost identical for both scanners, with Cohen's kappa ranging from 0.80 to 0.86 for different pathologists and different scanners. Independently, the images were also assessed by two experienced senior pathologists. Cohen's kappa of COMPASS was comparable to the Cohen's kappa for two senior pathologists (0.79 and 0.68).

Radiologists' Performance at Reduced Recall Rates in Mammography: A Laboratory Study

Rationale and objectives: Target recall rates are often used as a performance indicator in mammography screening programs with the intention of reducing false positive decisions, over diagnosis and anxiety for participants. However, the relationship between target recall rates and cancer detection is unclear, especially when readers are directed to adhere to a predetermined rate. The purpose of this study was to explore the effect of setting different recall rates on radiologist's performance. Materials and Methods: Institutional ethics approval was granted and informed consent was obtained from each participating radiologist. Five experienced breast imaging radiologists read a single test set of 200 mammographic cases (20 abnormal and 180 normal). The radiologists were asked to identify each case that they required to be recalled in three different recall conditions; free recall, 15% and 10% and mark the location of any suspicious lesions. Results: Wide variability in recall rates was observed when reading at free recall, ranging from 18.5% to 34.0%. Readers demonstrated significantly reduced performance when reading at prescribed recall rates, with lower sensitivity (H=12.891, P=0.002), case location sensitivity (H=12.512, P=0.002) and ROC AUC (H=11.601, P=0.003) albeit with an increased specificity (H=12.704, P=0.002). However, no significant changes were evident in lesion location sensitivity (H=1.982, P=0.371) and JAFROC FOM (H=1.820, P=0.403). Conclusion: In this laboratory study, reducing the number of recalled cases to 10% significantly reduced radiologists' performance with lower detection sensitivity, although a significant improvement in specificity was observed.

Observer Variability in Breast Cancer Diagnosis between Countries with and without Breast Screening.

RATIONAL AND OBJECTIVES: Image reporting is a vital component of patient management depending on individual radiologists' performance. Our objective was to explore mammographic diagnostic efficacy in a country where breast cancer screening does not exist. MATERIALS AND METHODS: Two mammographic test sets were used: a typical screening (TS) and high-difficulty (HD) test set. Nonscreening (NS) radiologists (n = 11) read both test sets, while 52 and 49 screening radiologists read the TS and HD test sets, respectively. The screening radiologists were classified into two groups: a less experienced (LE) group with ≤5 years' experience and a more experienced (ME) group with ≥5 years' experience. A Kruskal-Wallis and Tukey-Kramer post hoc test were used to compare reading performance among reader groups, and the Wilcoxon matched pairs tests was used to compare TS and ND test sets for the NS radiologists. RESULTS: Across the three reader groups, there were significant differences in case sensitivity (χ2 [2] = 9.4, P = .008), specificity (χ2 [2] = 10.3, P = .006), location sensitivity (χ2 [2] = 19.8, P < .001), receiver operating characteristics, area under the curve (χ2 [2] = 19.7, P < .001) and jack-knife free-response receiver operating characteristics (JAFROCs) (χ2 [2] = 18.1, P < .001). NS performance for all measured scores was significantly lower than those for the ME readers (P < .006), while only location sensitivity was lower (χ2 [2] = 17.5, P = .026) for the NS compared to the LE group. No other significant differences were observed. CONCLUSION: Large variations in mammographic performance exist between radiologists from screening and nonscreening countries.

Benefits of Independent Double Reading in Digital Mammography: A Theoretical Evaluation of All Possible Pairing Methodologies.

RATIONALE AND OBJECTIVES: To establish the efficacy of pairing readers randomly and evaluate the merits of developing optimal pairing methodologies. MATERIALS AND METHODS: Sensitivity, specificity, and proportion correct were computed for three different case sets that were independently read by 16 radiologists. Performance of radiologists as single readers was compared to expected double reading performance. We theoretically evaluated all possible pairing methodologies. Bootstrap resampling methods were used for statistical analyses. RESULTS: Significant improvements in expected performance for double versus single reading (ie, delta performance) were shown for all performance measures and case-sets (p ≤ .003), with overall delta performance across all theoretically possible pairing schemes (n = 10,395) ranging between .05 and .08. Delta performance for the 20 best pairing schemes was significant (p < .001) and ranged between .07 and .10. Delta performance for 20 random pairing schemes was also significant (p ≤ .003) and ranged between .05 and .08. Delta performance for the 20 worst pairing schemes ranged between .03 and .06, reaching significance in delta proportion correct (p ≤ .021) for all three case-sets and in delta specificity for two case-sets (p ≤ .033) but not for a third case-set (p = .131), and not reaching significance in delta sensitivity for any of the three case-sets (.098 ≥ p ≥ .067). CONCLUSION: Significant benefits accrue from double reading, and while random reader pairing achieves most double reading benefits, a strategic pairing approach may maximize the benefits of double reading.